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ABSTRACT 

A computer-assisted language learning (CALL) software 
evaluation form that looks directly at important aspects of language 
teaching approaches is presented. Examples of typical evaluation 
checklists are described to illustrate how they fail to address the 
appropriate questions for language teaching software. Three major 
categories of language teaching approaches (behaviorist , explicit 
learning, and acquisition) are defined and discussed as to how the 
principles can be applied to software evaluation. Ways in which to 
evaluate software for its potential use in teaching strategies for 
learning are discussed, and the relationship between software, 
approach, and syllabus is considered. A procedure is outlined for 
using the form to evaluate CALL materials, and implications of these 
concepts for software design are suggested. (CB) 
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Introduction 



Evaluating software for computer-assisted language learnine fCALU is a 
for^dable task, even for those who are relativdJUilini£l^;¥.eld 
wtiether one cames out an evaluation informally for a local institution or 

nrT'" ^ P""'^'*^^ ^« least five pi^me- 

ters of the judgmental process that make the evaluation of courseware 
more difficult and challenging than that of conventional textClT^ 
First, there is the sheer novelty of this infant field of instructional' tech- 
nology to contend with. Reviewers of textual materials generally have had 
T^Lrr °^ >vith print madia as student and 

UteSte Ind^S^fi m"^ ''^'^ evaluators may be only recently computer 
S ^ ^ l^ast more than a few 

years ot expenence using computer materials in their classes; fewer still 
have ever experienced CALL from the students' perspective. 

textS LliiTif "'"^'^ °° '° ^'^^''e^ ^'ft^are as with a 

U S ^ *f materials. With few exceptions, the evaluator 

Vt) must proceed relatively lockstep once a choice is made from the mafn 
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menu. This restrictioia makes it very difficult to obtain an overview of the 
lesson format and content as quickly as is possible with a textbook. 

Third, the relative complexity in the placement of lesson components 
on a disk makes the review process laborious. The hierarchical structuring 
and branching routines typically found in some of the more complex pro- 
grams may deflne a large number of possible paths through a given lesson, 
which makes it difficult to view all of the material that a student might 
access while using the disk. 

Fourth, there are visual and auditory dimensions in computer software 
missing in print material and f hat require evaluation. In the visual dimen- 
sion, while both a textbook and computer software may include text alone, 
text plus pictures, or diagrams and charts, only the computerized lesson 
can animate graphics and highlight aspects of the text in a dynamic way. 
And speech synthesis aside, there are the congratulatory and critical 
buzzes, beeps, and electronic melodies that require a judgment whether 
they actuaUy enhance or detract from the lesson. 

Finally, there are interactional aspects to consider. To what extent does 
the student control the lesson or vice versa? How ^^intelligently** does the 
program evaluate the student's input? How does the computer r<;spond 
after having made that evaluation? A program — whether drill and prac- 
tice, tutorial, or simulation — in a real sense acts as teacher; the evaluator 
must determine to what extent that '^teacher*' is able to aid the student in 
learning the taiget language. 

The relative newness of CALL software has meant that most guidelines 
or checklists for software evaluation concentrate lai^gely on the parame- 
ters mentioned above, along with the equally vital questions about what 
hardware is required to run a given piece of software. As useful as such 
evaluation forms may be, however, they typically fail to address the crucial 
questions related to approach^ the essential foundation of all language 
instructional material. While the variables mentioned above are impor- 
tant to evaluate^ the bottom line must be "^How is this piece of software 
going to improve a student's proficiency in the taiget language?** 

This chapter discusses a supplementary evalution form developed to 
aid in answering this question by looking directly at important aspects of 
language-teaching approaches, where '^approach'* is taken to mean the set 
of underlying principles that outline a set of conditions for successful lan- 
guage learning and that, in turn, often follow from a theory of language 
acquisition and provide the foundation for specific classroom methods 
and techniques (Anthony, 1). Thus, the first section in this chapter shows 
how a typical checklist fails to address the appropriate questions for 
language-teaching software. The second section defines three major cate- 
gories of language-teaching approaches and suggests that the principles 
they encompass can be applied to software evaluation. The third section 
discusses the evaluation of software for its potential in teaching the stu- 
dent strategies for learning, and the fourth will consider the relationship 
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between software, approach, and syllabus. The final section introduces 
the supplementary evaluation form and outlines a procedure for using it 
to evaluate CALL materials. The chapter concludes with some implica- 
tions of these concepts for software design. 



Problems with Existing Checklists 

There are a number of published software evaluation checklists, but 
many, such as the MicroSIFT form produced by the International Council 
for Computers m Education (Marler, 3 1), have been designed for use with 
CAI matenals m any field. Using them to review foreign language materi- 
als assumes that learning a second language is essentially the same as other 
types of learnmg when, in fact, it is not. However strongly psychologists 
linguists, and language teachers may disagree on other issues, the over- 
whelming majonty support the view that learning a second language 
differs in significant ways from learning anything else. 

In addition to such general-use forms and questionnaires, other 
checklists that have been developed specifically for second and foreign 

S?f n^?''"' ' ^' "°P^ « 23, and Curtin and 

ShiMU, Chapter 10 in this volume) similarly do not focus on the relative 
tit of the software to instructional approach and require supplementation 
to extract meaningful judgments whether the materials are likely to 
accomplish their advertised ends. The checklist formulated by CALICO 
(currently under revision) is a case in point, as can be seen in Figure 1 (an 
abridgment by this author of the original in which contents irrelevant to 
language-teaching approach considerations have been omitted and the 
Items renumbered for internal consistency). 
I. General Questions 

1. Is the courseware intended to be teacher dependent or 
teacher independent? 

2. Are the exercises mechanical, meaningful, or communicative? 
II. Pedagogical Considerations 

1 . Are the courseware contents substantively correct? 

2. Are the explanations complete and adequate in number? 

3. Are there adequate examples? 

4. Are concepts presented well? 

5. Can the courseware be used to introduce material as well as 
^reinforce it? 

6. Are there adequate application activities? 

7. Are there adequate evaluation activities? 

«. Are all instructional units ofapproximately the same length? 
Are all the instructional units at approximately the same level 
of difficulty? 
10. Is the purpose of the package well defined? 
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1 1 . Does the package achieve its deflned purpose? 

1 2. Is the package motivational? 

1 3. Is the level of difficulty found in the courseware appropriate 
to the target audience? 

14. Is the courseware suitable to students with a wide range of 
ability levels? 

15. Does the level of difficulty vary according to the demon- 
strated ability level of the student? 

III. Adaptability to Computer Medium 

1. Is the instructional program interactive? 

2. Is it clearly individualized? 

3. Are branching or help options provided for students who 
need remedial attention? 

4. Does the student have more than one chance to answer the 
quest, jn correctly? 

5. Are opportunities provided for student feedback? 

6. Can the student return to the start at any time? 

7. Can the student exit at any time? 

8. Does the student control the rate of presentation? 

Figure 1. Abridged Version of the CAUCO Software Evaluation Form 

While lack of space prevents looking zt each item in Figure 1 , a signifi- 
cant weakness throughout is the omission of the word languagCy which 
detracts significantly from the face validity of the instrument in evaluat- 
ing language materials. The CALICO form thus is just as suitable for 
evaluating CAI materials in chemistry or mathematics as it is forjudging 
language-teaching courseware. A second problem with the CALICO 
checklist is a bias toward what can be called an explicit learning approach, 
considered by some to be the dominant form of second-language teaching 
at both the high school and university levels, although perhaps not the 
most effective. 

In summary, there are shortcomings in the majority of existing 
checklists for evaluating CALL software. This article will suggest guide- 
lines to overcome these limitations, taking as a basis for intelligent 
evaluation of CALL courseware the need for the evaluator to have a clear 
understanding of the approach underlying the curriculum and the syllabus 
for which it is intended. 



Three Categories of Approach • 

Over the years, a number of distinct approaches to second-language teach- 
ing have guided classroom efforts and materials to help students learn. 
Although the boundaries that distinguish them may not always be clear. 
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three major categones of approach can be identified for discussion pur- 
poses. Unhke the case of a specific approach, however, none should be 
taken as corresponding directly to any particular theory; rather, they are 
simply convenient labels for discussing distinguishable trends in second- 
anguage learning and teaching. Some of the more recently proposed 
theoretical models of second-Ianguage learning— Kiashen (26), Bialystok 
(6,7) Strevens (43), Swain (44>~rind representation under aU the head- 
ings, while others—stimulus-response theories underlying behaviorist 
approaches (e.g.. Brooks, 8)— are more clearly limited to a single 
descnptor. Nevertheless, these three categories reflect useful distinctions 
for matenals development and CALL software evaluation, since they 
reflect major pnncipled components of specific theories and models of 
second-language acquisition (see also Chapter 7 by Doughty in this 



Behaviorist Approsches 

A number of authors (Dalgish, 16, Ariew, 2, Underwood, 46. Baker 4) 
have noted that much of currem CALL software seems to be based on a 
stimulus-response theory of language learning. Whether one agrees with 
this type of approach or not, the characteristics that distinguish behavior- 
ism from other approaches are worth noting so Uiat they can be recognized 
when they appear in CALL courseware. "'snizea 
HistoricaUy, behaviorist approaches to language learning are based on 
me principle that a response, linguistic or otherwise, is learned behavior 
resulting from associating that response with a given stimulus. Through 
positive reinforcement for conect behavior and negative reinforcement 
tor incorrect behavior, these responses become overleamed until they are 
automatic. The main difference between learning a language under a 
behavionst approach and learning mathematics, forexample, istiiat most 
oi the methods denved from it (such as audiolingualism) also place heavy 
stress on learning about the second-language culture. 

Larsen-Freeman (28) lists a number of principles underlying the 
audiolingual metiiod. Considering just tiiose most relevant to tiie com- 
puter medium. CALL software will be representative of a behaviorist 
approach to language teaching to tiie extent tiiat it does tiie foUowing: 

1. presents vocabulary and stmcture appropriate to tiie learner's level 

2. maintains the learner's attention to task 

3. does not accept errors as correct answer 

4. requires the learner to input tiie correct answer before proceeding 

5. provides the learner witii positive feedback for correct answers 

6. provides sufficient material for mastery and overleaming to occur 

7. reinforces patterns and vocabulary^resented in a lesson 
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8. presents grammar rules or patterns inductively with no attempt at 
teaching explicit formulations of them 



Explicit Learning Approaches 

Explicit learning approaches, including the so-called cognitive 
approaches in second-language teaching trace their heritage to the gram- 
mar and grammar translation methods of the Middle Ages and 
Renaissance. The central notion of explicit learning is that the target lan- 
guage can be learned through a conscious knowledge of the meanings of its 
words and the rules of its grammar. Currently, there are several theoretical 
positions taken with respea to explicit learning. Some researchers (Ellis, 
19; Bialystok, 6,7) maintain that explicit learning and practice of gram- 
mar rules can lead to the automatic processing necessary for fluent 
conversation in the target language. Others, including Krashen (26), 
maintain that explicit learning cannot lead to automatic processing and 
that conscious linguistic knowledge can only be utilized by the Monitor, a 
cognitive linguisticdevice that applies rules cons^^iously in speech produc- 
tion and edits the output at a significantly slower rate than the automatic 
processing system. 

A somewhat different position is taken by McLaughlin et al. (32), who 
propose an information-processing approach that distinguishes con- 
trolled from automatic processing but assumes that either can be 
conscious or unconscious. In short, McLaughlin et al. allow for explicit 
learning to become automatic under certain conditions. Even within 
Krashen*s Monitor Theory, however, it is acknowledged that explicit 
learning can be utilized when time is not a significant factor, as in writing 
or intensive reading. Further support for explicit learning approaches in 
teaching comes from a review and reanalysis by Long (29) of data from 
twelve experiments in language learning. Long concluded that explicit 
instruction at least facilitates the development of language proficiency 
and is more effective than language exposure alone. 

If explicit learning is aimed at both consciously learned rules and their 
appropriate application in comprehending and producing the target lan- 
guage, i* is desirable for the learner to have access to as much relevant 
information about the rules as possible. Thus, even in a standard practice 
exercise, whether mech^ical or meaningful, it is helpful for students to 
have thir option of requesting assistance in the form of hints to lead them 
to a correct response as well as answers to questions like *^Why is my 
answer wrong?" and '*Why is this answer right?** Few teachers in a typical 
classroom situation fail to respond to such questions, yet many current 
computer programs presumably aimed at explicit learning do not make 
this kind of assistance available. A related issue is the importance of 
accepting a range of appropriate answers for certain exercise items. If the 
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goal of- the exercise is to move the student in the dirertion of native- 
speaker competence, then any learner language a native speaker would 
accept as correct in a given context also should be considered correct 
Teachers typically accept a range of answers as correct in classroom or 
homework exercises (often accompanied by a comment about minor dif- 
ferences in meaning or register); not many CALL programs, however, are 
designed to accommodate this need. 

^ Providing drill-and-practice material in explicit learning approaches 
that IS meaningful, contextualized, and interesting to the students is a 
recent trend that parallels developments in communicative approaches 
(Madsen and Bowen, 30). In much the same vein. Oiler (36, p. 49) advo- 
cates paying greater attention to "story-writing" principles in ESL 
teaching, suggesting that while teachers should maintain a commitment 
to the explicit learning of grammar, they should also look at the orae- 
matic factors that affect the meaningfulness and comprehensibility of 
discourses used for instructional purposes. Although the focus in explicit 
learning matenals is still more on form than on language use there are 
compelling arguments in support of this trend. An exercise that is mean- 
ingful requires the students to integrate grammatical form and lexical 
content, leading to a deeper level of psycholinguistic processing. An 
exercise that is appropriately contextualized (e.g., a coherent parawaph 
or a senes of sentences relating to a common theme, a picture, etc ) wiU 
I^d the students toward the pragmatic competence necessary to extend 
the niles or vocabulary items coirecUy to novel instances. Finally, mate- 
nal that is interesting is more likely to hold students' attention and 
motivaie them to complete the exercise successfully and attempt others 



s.vrf?t;^^T! ? *\^'P,^5 ^''P^'"* learning paradigm that relates exclu- 
sively to CALL matenals m volves student control of the software. Higgins 
(21) has argued that student-controUed software wiU be far more eff^ve 
than lockstep program-controUed learning. Research by Stevens (41) suu- 
ports this view, although the optimal degree of student control wiU no 
doubt vary with the student's level and the task. A study by Hubbard et al 
U5) also bears on this issue by pointing out that if students are given a sie- 
mficant amount of control without appropriate training in how to use the 
software, they may fail to utilize the powerful options available to them 
iranslating the above considerations into evaluation criteria for CALL. 

S'^LThd^'SS^ '» 

1. introduces or reviews grammar rules and word meanings in an under- 
standable, learnable, and reasonably accurate form 

2. provides effective practice so that (a) novel target-language input can 
be readily understood, and (b) the learner's understanding of rules 
leads to the production of grammaticaUy acceptable spoken or writ- 
ten tai^et-language discourse^ novel situations 
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3. gives meaningful rather tlian mechanical practice 

4. gives practice contextualized in a coherent discourse larger than a sin- 
gle sentence 

5. provides hints of various types to hdp lead students to acceptable answers 

6. accepts alternative correct answers within the given context 

7. provides the student with explanation of correct answers 

8. anticipates incorrect or inappropriate answers and explains why such 
answers are incorrect or inappropriate 

9. maintains the student*s interest throughout the exercise 
1 0. allows an appropriate degree of student control 

In short, software that follows the principles of explicit learning as out- 
lined above should be essentially tutorial in its approach and based on 
paragraph-length, episodically related examples of realistic language 
rather than isolated sentences. In programming the lesson, the author 
must anticipate the kinds of information that would most help the student 
learn and retain the material and provide that information in a form that 
is accessible and understandable. Drill-and-practice software, especially 
the type whose exercises can be processed in a superficial way, is simply 
less likely to lead to the learning and retention of explicit rules and vocab- 
ulary than is software that requires a deeper level of cognitive processing. 



Acquisition Approaches 

While the learning/acquisition distinction has only been popularized 
recently through the work of Krashen (26) and Dulay, Burt, and Krashen 
(18), the idea that languages can be learned effectively without formal 
study of language stnicture and vocabulary has been with the field of 
second-language learning for some time. To some extent, the Direct 
Method and other implicit learning approaches are based on this idea. 
The current revival of the notion of natural language learning can be 
traced to Newmark (34) and Newmark and Reibel (3S), who suggested 
that languages can be learned in a natural environment that holds mean- 
ingful and understandable ^'chunks'* of language provided that the learner 
simply pays attention to the language as it is being used. Krashen (26) lat^ 
refined these ideas Into a theory of second-language learning and acquisi- 
tion involving (1) the Monitor, mentioned in the previous section, for 
explicit learning and processing, (2) an Organizer, or Language Acquisi- 
tion Deviije, and (3) an Affective Filter (Dulay, Burt, and Krashen, 18). 
The Organizer and Affective Filter are central to the acquisition portion 
of the overall model, as they are involved in the learner's unconscious 
analysis of strings of speech in the processing of speech input, the produc- 
tion of output, and the synthesis of new rules. 
Krashen*s model, though far from being universally accepted, has 
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exerted a profound influence on second-language acquisition research in 
the past decade. It has also provided a theoretical foundation for a class of 
communicatively oriented approaches (TerreU, 45; Krashen and TerreU, 
27; Winitz, 49), There is, in fact, a significant overlap between acquisition 
approaches and communicative approaches in general, even though the 
two are not synonymous. Specifically, it is possible to include communi- 
cative methodology within the umbreUa of an explicit learning approach. 
As Brumfit (10, p. 37) has pointed out, however, a number of second- 
language acquisition researchers and language-teaching methodologists 
have assumed a dichotomy roughly parallel to the learning/acquisition 
division in Krashen's theory and, while they vary somewhat with respect 
to the role explicit learning might play in a communicative approach, the 
importance of acquisition in developing communicative competence is 
generally supported. For this reason, in the discussion that follows com- 
municative approaches are subsumed under ••acquisition.*' (See Doughty, 
Chapter 7 in this volume, for a further discussion of the acquisition/ 
learning distinction.) 

The first requirement for acquisition in Krashen's model is that the 
input (the language directed at the learner) be comprehensible, i.e., that its 
meaning can be determined by the learner from the communicative con- 
text and the linguistic information in the learner's acquired system at that 
point in the acquisition process. Given some linguistic input, the Affec- 
tive Filter determines how much of this input makes its way to the 
Onganizer for processing. Two significant variables in the Affective Filter 
are the learner's attitude and motivation. A positive self-image, a low level 
of anxiety, and a strong desire to learn the language (and engage in the 
communicative tasks necessary for acquisition) are among the criteria for 
teeping the Affective Filter at a low level, thus allowing maximum linguis- 
tic input into the Oiganizer. Finally, the learning environment and the - 
types of learning tasks are assumed to have a significant effect on the 
Affective Filter. 

Krashen's Oiganizer is reminiscent of the Language Acquisition Device 
that Chomsky ( 1 3) hypothesized for first-language acquisition. Input that 
has not been filtered out, or intake as Krashen calls it, is unconsciously 
analyzed by the Oiganizer, which also formulates rules (again uncon- 
sciously). While the internal operation of the Organizer remains a 
mystery, it is hypothesized to operate most effectively on linguistic struc- 
tures that are just beyond the level already attained in the acquired 
grammar. Because of the implied developmental sequence in the acquisi- 
tion of grammatical rules, it is also generally assumed that overt error 
correction will have no real value in acquisition and, in fact, may be detri- 
mental, because correcting learner errors leads to anxiety and raises 
affective barriers. 

Krashen's model focuses on the learner's linguistic input and does not 
require any production expUcitiy i|i pj'der for acquisition of linguistic and 
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communicative competence to occur. However, acquisition-based 
approaches generally put some emphasis on developing the learner's abil- 
ity to produce as well as to understand novel utterances. Thus, a final 
point about acquisition approaches is that they aUow the learner practice 
in producing comprehensible output in addition to processing input. 

Although Krashen*s model has had little influence on the development 
of CALL software, it is not the case that CALL specialists have ignored 
acquisition approaches altogether. Higgins and Johns (22), Dalgish (16), 
and Baltra (S) discuss the use of games and simulations as communicative 
activities and stress the value of learner-learner conversation in front of 
the screen. Having pairs and even larger groups of learners working 
together at a single screen allows the computer material to act as a catalyst 
to promote real communicative interaction between the participants. 
Underwood (46), another advocate of communicative CALL, discusses at 
length the need for developing software that is consistent with current lin- 
guistic and language-learning theory. His thirteen premises for communi- 
cative CALL have much in common with the evaluation criteria that are 
presented below. Unlike these evaluation criteria, however, many of 
Underwood's premises focus on what communicative CALL is not: 
'^Communicative CALL will never try to do anything a book could do just 
as weir (p. 54), while some of his other premises are more consultatory: . 
•^Communicative CALL will use the tai:get language exclusively** (p. 53). 

Looking at the above factors in terms of computer courseware, and fol- 
lowing Krashen*s theoretical model with the addition of some insights 
from communicative methods, CALL software will be representative of 
an acquisition-oriented approach to the extent that it does the following: 
L provides meaningful communicative interaction between the learner 

and the computer 

2. provides comprehensible input at a level just beyond that currently 
acquired by the learner 

3. promotes a positive self-image in the learner 

4. motivates the learner to use the software 

5. motivates the learner to learn the language 

6. provides a challenge but does not produce frustration or anxiety 

7. does not include overt error correction 

8. allows the learner the opportunity to produce comprehensible output 

9. acts efTecti vely as a catalyst to promote learner-learner interaction in 
the target language 

Note that "meaningful communicative interaction** (number 1 above) 
does not necessarily imply that both the learner and the computer use lan- 
guage in an exchange. It is possible for the computer to react to input 
nonlinguistically (e.g., by moving text or graphics in response to the 
leamer*s command) and for the learner to react to the computer*s target- 
language output nonlinguistically (e.g., by moving the cursor or a graphics 
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figure with pointing devices, such as the cursor keys or a mouse in 
response to the computer's command). In fact, the latter type of interac- 
tion is typical of beginners in delayed production approaches and 
methods such as the.Natural Approach (TerreU, 45; Krashen and TerreU, 
27), Total Physical Response (Asher, 3), and the Comprehension 
, Approach (Winitz, 49). 

This section has focused on three categories of approaches— 
behavionst, explicit learning, and acquisition— which were defined 
broadly in the interest of comprehensiveness. These broad categories of 
definiuon are not meant to imply, however, that all approaches will fit 
directly under one heading; rather, a given approach may not share aU of 
the principles of its closest superordinate category and may include other 
significant pnnciples not mentioned above (such as whether the native 
language may be used at aU in instrurtion). In practice, too, the individual 
teacher may quite consciously vary the approach, based on perceived stu- 
dent needs. For example, in second or foreign language classes or 
programs whose goals include grammatical accuracy as weU as communi- 
cative competence (as is often the case with languages taught for academic 
or professional purposes), it is not uncon3S;«7,on to see a preference for 
acquisition approaches in conversation/«i..^ussion components and 
explicit learning approaches in reading/writing/grammar components 
onen, there is vanation based on the learner's level as weU, with acquisi- 
tion approaches favored for begsnning levels and explicit learning 
approaches favored for more advanced ones. Nevertheless, the parame- 
ters listed above for these categories are useful aids in determining the 
language-teaching approach underlying given CALL software and in 
turn, provide a means forjudging a fundamental question in softv^e 
evaluation: How well does the teaching approach manifested in the soft- 
ware match that of the teacher whose students will be using it' 



Learner Strategy Orientation 

Anadditional area of consideration in judging the probable effectiveness 
ot foreign language software is the degree to which the materials may 
directly or indirectly promote the use of particular strategies in the 
learner, that is, procedures for learning, acquiring, or using the target lan- 
guage more easily and eflectively. The concept of learner strategies is not 
limited directly to any particular category of approaches. In fact, it is pos- 
sible for the learner to be taught or to develop strategies compatible with 
approaches in any of the three major categories mentioned above. 

The idea of using strategies to enhance learning is not new. Learner 
strategies have received increasing attention from both researchers in 
||S^nd-language learning and materiak developers in the past decade as 
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part of the general trend toward focusing on the language learner, and sev- 
eral taxonomies have been proposed to describe them. Most, in one way or 
another, distinguish direct or cognitive learner strategies (those involving 
deliberate manipulation of material to enhance learning or retention) 
from indirect or metacognitive ones (those involving self-monitoring, 
self-assessment, and goal setting— Oxford-Carpenter, 37, p.l). Wenden 
(47) has added communication strategies (those used to facilitate infor- 
mation exchange) and global practice strategies (those that lead the 
learner to utilize the environment effectively for target language practice). 
Rather than attaching itself to any particular taxonomy, however, the dis- 
cussion that follows will look at learner strategies from the viewpoint of 
some general principles that are particularly appropriate for the evalua- 
tion of strategy-oriented software. 

In a sense, any structured or semistructured exercise, from pattern 
practice to open-ended role play, represents a strategy on the part of the 
teacher for aiding the student in gaining proHciency in the tai^et language. 
But these are really /eacA/Vi^ strategies. A /ear/ier-strategy orientation, on 
the other hand, involves focusing more on those strategies that the learner 
may come to employ consciously and control independently. For example, 
in teaching vocabulary with a learner-strategy orientation, the focus is not 
on learning individual lexical items; instead, the teacher introduces and 
provides meaningful practice in strategies for guessing the meaning of an 
unknown word (attending to context, deducing the part of speech, decom- 
posing it into stem and derivational morphemes, etc.). In the first 
instance, for example, the student can be taught the strategy of looking 
ahead to the words that follow the unknown word for clues to its meaning, 
rather than simply ignoring it or stopping and reaching for the dictionary 
as soon as it is encountered. As this example illustrates, the teaching of 
strategies helps move the learner from being a passive recipient of linguis- 
tic input to becoming a more conscious and autonomous processor; in 
short, learner strategies shift much of the responsibility for successful 
learning from the teacher to the student (Wenden, 47, p. 5). 

It is significant for CALL that many of the strategies that are useful in 
second-language learning are utilized by native speakers as well. This is 
because many learner strategies are aimed primarily at improving the 
individual's performance, rather than achieving linguistic competence. 
Thus, CALL software developed for native speakers to enhance their strat- 
egies for learning, particularly in the areas of reading and writing, may be 
quite helpful for advanced learners of a target language. 

As with other strategy-oriented material, the focus for CALL software 
should be on strategies that can be learned consciously or induced in the 
learner, rather than on those that seem to be a universal by-product of 
second-language learning, e.g., overgeneralization and avoidance as dis- 
cussed by Brown (9). Strategies to be taught may range from the very 
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general, e.g,, the fourteen major ones for •*the good language learner** dis- 
cussed by Rubin and Thompson (38)— learn to tolerate ambiguity, let 
context help you, learn formalized routines, etc.— to the very detailed and 
skill-specific, e.g., skimming a passage by reading the first sentence of 
each paragraph to gain a general idea of content (Grellet, 20). 

There are several types of strategies that seem particularly well-suited 
to being introduced and practiced on the computer. In reading, for exam- 
pie, psycholinguistic research has pointed to the importance of top-down 
processing strategies, using information from skimming a passage, from 
its title and introduction, or from the reader*s background knowledge of 
the subject to build an anticipation schema for its rhetorical structure 
and content (Coady, 14; Carrell and Eisterhold, 12). Experiments 
reported in Carrell (11) provide evidence that ESL learners even at the 
advanced level make less use of such strategies than do typical native 
English readers, and it is plausible to assume that this may be true of sec- 
ond and foreign language reading in general. In writing, there ace 
production strategies such as writing dialogs, brainstorming, list mak- 
ing, and flexible outlining that many second-language learners are either 
unaware of or ignore (Spack, 40; Hubbard, 24). At the level of rhetorical 
structure, Scarcella (39) has shown that the reader orientation strategies 
of ad vanced ESL writers differ significantly from those of native writers, 
which suggests that strategies learned in the first language may not trans^ 
fer to or be appropriate in a second. 

Unlike the behaviorist, explicit learning, or acquisition approaches, 
there are no specific learning theories or models associated with learner 
strategies. Consequently, much of what foUows is based on the writer's 
views of how to teach learner strategies effectively. 

A number of factors must be considered in producing materials to teach 
learner strategies. A particular strategy will be effective to the degree that 
It fits both the learner's needs and his or her preferred learning style. If the 
strategy is being taught expliciUy, as is generally the case, it must be pre- 
sented in a comprehensible way and be accompanied by an explanation of 
the principles underlying it— without this, the learner is unUkely to be 
convmced of its value. Reinforcement tasks, then, should be designed so 
that they are accomplished more efficiently if a given strategy is used 
appropriately In many cases, a variety of related strategies may be pre- 
sented together with explanations of their respective strengths and 
weaknesses. In this instance the accompanying exercises will give the 
learner the opportunity to experiment and discover which strategies work 
best under specific conditions. 

Based on the preceding considerations, CALL software will effectively 
I^omote the learning and use of learner strategies to the extent that it does 
the following: 

1 . introduces the learner to strat^es that are useAil and immediately usable 

2. introduces the learner to strategies appropriate to the learner's level 
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3. explains the value of the strategies 

4. provides meaningful practice in the use of the strategies 

5. presents practice material in such a way that the task is more easily 
or successfully accomplished if the appropriate strategy or strategies 
are used 

6. provides, when possible, a variety of strategies (or of techniques for uti- 
lizing a given strategy) for a given type of task suited to a range of 
learning styles 

7. provides feedback on which strategies might have worked best for 
given tasks after the learner has attempted them 

As research continues in the area of learner strategies, it is likely that more 
explicit methods and techniques will be developed to teach and to judge 
their relative effectiveness. In the meantime, the parameters mentioned 
above provide a usable evaluation metric for determining the potential 
effectiveness of strategy-oriented software. 



The Syllabus and CALL Materials 

The software evaluator, particularly one making decisions for his or her 
class or program rather than for published review, must determine to what 
extent the orientation and content of the software is compatible with the 
orientation, content, and sequencing of an institution-specific syllabus. 
Some of the parameterr i'nat are useful to consider in answering that ques- 
tion are discussed below for four major syllabus types — structural, 
situational, notional-functional, and content-centered — recognized gen- 
erally as typical in second or foreign language teaching. 



Structural Syllabus 

The focus of a structural syllabus is the grammar of the target language. 
Within a behaviorist or explicit learning approach, the grammar patterns 
or rules of the language are typically sequenced along continua of pre- 
sunied ease of learning (easiest structures first) and to a lesser extent 
relative frequency (more common structures first). In either case, the 
stnictur il focus of CALL software to be used as an integral pait of a course 
must be at a level that is consistent with that prescribed by the syllabus. If 
the software is used on a voluntary basis by the students, then it may be 
appropriate to have material at a lower level for remedial purposes. 

A learner-strategy orientation in CALL can be incorporated into a 
structural syllabus to the extent that the software is designed to introduce 
and practice general procedures for learning and using grammatical con- 
structions. As reading is often a conmonent of an explicit learning 
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approach in a structural syUabus. software that focuses on appropriate 
reading strategies can also fit into this combination. 

The structural syUabus per se is not reaUy compatible with an acquisi- 
tion approach, although it is possible to have genuine communicative 
exercises, either on the computer itself or involving the learners in conver- 
sation with the computer acting as a catalyst, which lend themselves 
naturally to promoting the use of a particular grammatical construction 
(e.g., giving instructions would promote the use of imperative forms) 



Situational Syllabus 



In a situationa^^ syUabus, the focus is on introducing the learners to situa- 
tions they are hkely to encounter in using the target language (ordering a 
meal, cashing a check) and helping them to buUd frames of reference for 
communicating appropriately. In practice, a situational syllabus is often 
integrated with a structural or notional-functional syllabus. However tiie 
central focus in a true situational syllabus is on tiie relevant vocabulary 
and the interchanges between tiie participants witiiin a prototypical con- 
text^ Ideally leading tiie learner to an understanding of tiie expectations 
both linguistic and nonlinguistic, of target language speakers. Regardles^ 
of the approach, CALL software being reviewed for use within a 
situational syUabus should demonstrate content and situations which tiie 
learner is hkely to encounter in real life; ideaUy, it should be sequenced to 
reinforce or complei^.ent tiie situations tiie learner is exposed to in tiie text 
and classroom. 

When a behaviorist approach is applied to a situational syUabus tiie 
goal IS presumably to driU tiie learner in prototypical situations and Can- 
ants witii a simtiar cornmunicati ve structure. CALL software tiiat involves 
this tjjpe of dnU, particularly if it aUows for tiie repetition necessary for 
iiabit formation, would be consistent witii tiiis combination. 

If an explicit learning approach is linked to a situational syllabus CALL 
softw^e consistent with tiie combination would need to provide lists of 
vocabulary to be learned, examples of appropriate interchanges, and a 
discussion of what tiie structural and sociolinguistic elements of the inter- 
^anges are, along with practice in initiating the conversation and 
rKponding appropriately. Bearing in mind earlier comments about tiie 
J'''u"'1J'^^'^ explicit-learning software, hints and explanatory 
feedback should also be provided. An appropriately programmed interact 
live videodisc containing a common situation, such as learning the 
^guage and behavioral protocols of going to a restaurant, would be an 
example of exphcit learning witiiin a situational syllabus 
„,^° approach can be maintained in a situational syllabus by 

Ming CALL software whose content provides comprehensible input in tiie 
pyen situations and tiiat aUows tiijlgmer to interact m\h tiie computer 
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and/or other learners. Simulations of common situations, for example, 
could provide both communicative language practice and information 
about target culture expectations, e.g., getting from the airport to a spe- 
ciflc location in a strange city. 

Learner-strategy software within a situational approach is also a possi- 
bility. In this case, the focus of the soft>;;^e is on strategies for acquiring 
information about the given situation by observing or asking native 
speakers for assistance, and on strategies for identifying and repairing 
miscommunication in an acceptable manner according to the target cul- 
ture standards of behavior. 



Notioaal-nuictional Syllabus 

Strictly speaking, a notional syllabus divides the target language into 
semantic categories (greeting, apology, anger, etc.) while a functional sylla- 
bus focuses on language use (asking, expressing feelings, responding, etc.). 
In practice, they are generally combined into a syllabus built around using 
language within particular semantic categories — asking for assiistance, 
expressing anger, responding to an apology, etc. (Wilkins, 48). While it is 
possible to realize a behaviorist or explicit learning approach within a 
notional/functional syllabas, Uie focus on real language use that lies at the 
root of this syllabus orietii:.M.ion makes it much more compatible with 
acquisition approaches. 

CALL software that allows for real communicative interaction between 
the learner and computer or between two or more learners in front of the 
computer will be in line with an acquisition approach in a notional- 
functional syllabus to the extent that it meets the criteria for 
acquisition-oriented software in general and provides practice in using 
the notions/functions called for by the syllabus at the level it is being used. 
Software that reviews notions/functions from earlier lessons can provide 
acceptable material toward this end as well, as long as the lessons remain 
challenging enough to engage the learners* interest. However, a reviewer 
should be wary of software that might assume that the learners have mas- 
tered a particular notion/function (e.g., making polite requests) that in 
fact they do not control, even if the grammar and vocabulary are familiar 
to them. 

Learner strategies relate weU to a notional-functional syllabus, since 
some of th(: notions/functions typically taught (asking for clarification, 
restating, etc.) can be viewed as communicative strategies as well. The 
type of CAIX software that would be relevant is similar to that mentioned 
above for the situational syllabus, i.e., that which focuses on strategies 
that aid the learners in acquiring information, comprehending it, and 
making themselves understood. 
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Content-Centered Syllabus 



iL f^u^T^'^T'^ 'y"***"*' '^S^^S^ focus; rather, it is 

the tool the student learns to manipulate to get at information about the 
subject matter of the course (Mohan. 33). To a limited extent, content- 
centered courses have been around for years in the form of foreign 
language classes whose material relates to teaching about the taiget cul- 
ture. TypicaUy. however, such courses have tended to have structural 
sequencing as the overriding consideration; that is. content-cemered 
courees simply mject coment imo a structural syllabus. More recentiy 
building on the model of bUingual transition programs where certain sub- 
jerts are taughtm the child's second language, content-centered courses 
focusing on other topics have appeared in second and foreign language 
teaching(Qirtam 15). particularly in EngUshforacademicpi^g^ 
language for business classes in the United States. Structural 
situation^, and nouonal-functional aspects of the language may stiU b<^ 
considered within this more recent view of content-centered Liguage 
teaching, but they are of secondary importance. 

Lite the notional-functional syllabus, the content-centered syUabus is 
founded on the assumption that meaningful, natural use of language in a 
realistic commumcative setting will lead to increased proficienQrin the 
tai^et language (Mohan 33. p. 1). Thus, the content-centered syUabus is 
not really compauble with the drill-and-practice focus of behaviorist 
approaches or the language-analysis orientation of explicit learning 
approaches. It is. however, quite compatible with acquisition approach^ 
and a learner-strategy orientation. pproacncs 
Within an acquisition approach, there are three important considera- 
tions in evaluating content-centered CALL software. The first, which is an 
miportant consideraUon for the course as a whole, is to be sure that the 
subject matter is hnted to the needs and/or interests of the leamere so that 
they are motivated to use it. The second consideration is to determine to 
wnat extent the content of the software is consistent witii that of the rest of 

toe «^urse. If too much of the content is already familiar, learner^! wiU be 
le« motivated to attend to it; on the other hand, if the concepts are too 
chaUenging or too dependent on unfamiliar background information. 

theymaybecomefiiistrated. Finally, the linguistic level must beappropri- 
ate for the learnere* current degree of proficiency. If the vocabulary and 

ISL ''"'^ '*°8uage acquisition may occur, 

evai though the content may be mastered. Similarly, if there are too many 
unlaiown words or unfamiliar constructions, then the input may not be 
comprehensible enough to allow for much acquisition, and again, frustra- 
tion may result. 

f nJl^' Strategies that are most valuable to introduce and practice in a 
|«>ment-centered course, in addition to those mentioned above for a 

WMmM:'-'---- ■■■ ■/: ■>,• - ..■-//■■■IS- ■-■^ ;■■■ ■■^:/.. - ^ 
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notional-functional syllabus, are those that involve more efficient lan- 
guage production and comprehension. CALL software that promotes the 
use of reading-comprehension strategies is particularly appropriate, as is 
software that focuses on strategies for organizing compositions, assuming 
the overall curriculum and course requirements include a writing 
component. 

In summary, the focus of this section has been on the criteria for evalu- 
ating software vis a vis its relative consistency with respect to the course or 
program syllabus and its orientation. The three most important criteria 
seem to be the following: 

1. the approach manifested by the software is compatible with that of the 
syllabus type 

2. the level and sequencing of the linguistic content is appropriate for the 
course as determined by the syllabus 

3. the subject-matter content is appropriate for the goals of the course or 
program as determined by the syllabus and the presumed knowledge 
base of the learners 

The degree to which the software meets these criteria will aid the 
evaluator in determining whether the CALL materials can be integrated 
effectively into the syllabus as basic or supplementary material. 



A Supplementary Evaluation Form for Language Teaching 
Approaches — 

This final section of this chapter presents a three-part supplementary 
evaluation form for CALL software (Figure 2) that takes into account the 
characteristics of the three types of approaches and the learner-strategy 
considerations presented above, and describes a procedure for using it. 
Users adopting this form for in-house evaluation are encouraged to adapt 
and simplify it, selecting those categories considered most relevant and 
ignoring the rest. As the evaluation criteria represent to some extent the 
author's interpretation of each approach, it is also possible that an 
evaluator will want to expand or revise the form on the basis of a different 
interpretation of these central principles. 

It is also important to realize that a teacher using this form may not find 
his or her teaching approach described or encapsulated within any single 
category. The direct method, for example, as described by Larsen- 
Freeman (28), includes characteristics of behaviorist approaches (with 
respect to pointing out and correcting errors as they occur) and acquisi- 
tion approaches (with respect to providing comprehensible input and 
promoting learner-learner interaction). Thus, teachers who do not 
embrace a particular method exclusively are even more likely to find their 
assumptions about language learning included under more than one 
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sunSlmln V '"^''y subscnbe to. For the same reasons, using this 

Sof m2e thT ""^•"'y^g the software itself reflects character- 
istics ot more than one category. Thus, using the form will reveal n/th* 
approach m which a given piece of software^^ay fi^ h gJL'e^t coie 
spondence and most efficient use, and (2) a pattern of res^i acrS 
ca egones that describe the approach subscribed to by th JTea^her 

wJ i n .T'"^ '° ""^^ '^"^^ th^t rates highly in atf the areas 

withm a given category. Thus, the evaluation form wiUalso provide iS 

cTnrt^o'^extTr^^el'^^^^^^^^ 
n,enta,.Evaluatrnt:;:S^r^^^^^^ 

?o™, r^rS-^^ " f ^^^^^^^'^ '"^d in S^?nSon wft^o^ 
[erTrt^of'S^'fo M "^^^^ ^^^"'^ ^« firstlection of tie chJ" 
aoD^rch t ™ ^ ''"""P'" ^'^^'^^^ the three categories ^f 
approach-behaviorist, explicit, acquisition; Part II encomoasses 

tor other pedagogical considerations. M»c;»uunndire 



I. APPROACH CHECKUST 
PRINCIPLE DEGREE? 
Behaviorist Approaches 

1. Presents vocabulary and 
stnicture appropriate to the 

learner;s level 0I23CJ I2345CJ 

2. Maintains the leamer*s 

attention on the task 0 I 2 3 CJ I 2 3 4 5 CJ 

3. Will not accept errors as 

conectanswcrs 0I23CJ I 2345CJ 

4. Requires the learner to -^^t^^ 
input correct answer before 

proceeding 0I23CJ I 2345CJ 

5. Provides the learner with 
positive feedback for correct 

answers 0I23CJ I 2345CI 

0. Provides sufficient material 
for mastery and 

overlcamins to occur 0I23CJ I 2345CI 
/. SubscquenUy reinforces pat- 
terns and vocabulary pre- 

^ntcd in earlier lessons 01230 I 2345CI 
«• i^rcsents grammar patterns 
inductively without 
attempting to teach formu- 

laUonsofnUes 0I23CI I2345CJ 



HOW WELL DONE? COMMENTC 
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Explicit Learning Approaches 

1 . Introduces or reviews gram- 
mar rules and word mean- 
ings in an understandable, 
leamable, and reasonably 

accuratefonn 0123CJ 12345CJ 

2. Provides effective practice 
so that (a) novel target- 
language input can be read- 
ily understood and (b) the 
Ieamer*s understanding of 
the rules studied leads to the 
production of grammati- 
cally correct spoken or writ- 
ten target-language forms 0123CJ i234SCJ 

3. Gives meaningful rather 

than mechanical practice 0123CJ 1234SCJ 

4. Gives practice 
contextualized in a coherent 
discourse larger than a sen- 
tence 0I23CJ 12345CJ 

5. Provides hints of various 
types to help lead students 

to correct answers 0123CJ I2345CJ 

6. Accepts alternative correct 

answen 0123CJ 1234SCJ 

7. Provides explanations for 
why correct answers are cor- 
rect 0I23CJ 12345CJ 

8. Anticipates incorrect 
answers and provides expla- 
nations of them 0I23CJ 12345CJ 

9. Maintains the learner's 
interest throughout the 

exercise 0I23CJ 12345CJ 

1 0. Allows an appropriate 

degree of student control 0123CJ 1234SCJ 

CJ * cannot judge Q " not at all 1 * poorly 

3 - to a great S - excellently 
extent 

PRINCIPLE DEGREE? HOW WELL DONE? COMMENTS 

Acquisition Approaches 

1 . Prov ides meaningful com- 
municative interaction 
betMveen the student and the 

comoulcr 01 23CJ 12 34 5CJ 

2. Pro\ ides comprehensible 
input at a level just beyond 
that currently acquired by 

theleamcr 0I23CI 12345CJ 2^ 
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3. Promotes a positive self- 
image in the learner 0123CJ I2345CJ 

4. Motivates the learner to use 

0123CJ 12345CJ 

5. Motivates the learner to 

acquire tbelanguage 0123CJ 12345CJ 

6. Provides a challenge but 
does not produce fnistra- 

tionoranxiety 0123CJ 12345CJ 

7. Does not include overt error 

corrccUon 0123CJ 12 345CJ 

8. Allows the learner the 
opportunity to produce 

comprehensible output 0123CJ 12345CJ 

9. Acts effectively as a catalyst 
to promote learner-learner 
interaction in the taiget Ian* 

guage 01 23CJ 12345CJ 

n. LEARNER STRATEGY CHECKUST 

1 . Introduces the learner to 
strategies that are useful and 

immediately usable 0123CJ 12345CJ 

Z Introduces the learner to 
strategies appropriate to the 

leamer*slevcl 0123CJ 123450 

3. Explains the value of the 

strategies 0123CJ 1 23450 

4. Provides meaningful prac* 
tice in the use of the strate- 

0I23CJ 12345a 

5. Presents practice material 
in such a way that the task is 
more easily or successfully 
accomplished if the appro- 
priate strategy or strategies 

arcuscd 0123CJ 12345CJ 

6. Provides an appropriate 
variety of strategies (or of 
techniques for utilizing a 
given strategy) for a given 
type of task suited to a 

range of learning styles 0123CJ 12345CJ 

7. Provides feedback on which 
strategies might have 
worked best forgiven tasks 
after the student has 

attempted them 0123CJ 12345CJ 

CJ - cannot judge 0 « not at all 1 - pooriy 

3 - to a great 5 « excellently 
extent 

22 
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IIL OTHER PEDAGOGICAL CONSIDERATIONS 

1 . Based on your own interpreUtion of ibc ratings, whai approach or approaches docs this 
courseware most clearly represent? u- i_ . • 

2. How wcU does the software fit into the syllabus for the class or program for which it is 

Ihc approach manifested by the software compatible with that of the syllabus? 
b. Are the level and sequencing of H^^uistic content appropriate for the course as 
determined by the syllabus? 

c Is the subject-matter content appropriate for the goals of the course or program as 
determined by the syllabus and for the presumed knowledge base of the learners? 

3. Briefly describe the methods and techniques used; corrmient on how successfully they 
have been adapted to the computer medium. 

Figure 2, Supplementary Evaluation Fbna for lan guag e Teaching Considerations 



In the first two sections of the form (Approach and Learner Strata), a 
two-part, scaled-response format replaces the simple yes/tio format char- 
acteristic of the CALICO checklist and many others. The first part 
(Column 1 ) is a purely quantitative measure, scaled from 0 to 3, and repre- 
sents the degree to which a certain criterion is meU with 0 meaning "noi at 
all** and 3 meaning •no a great extent " The second part (Colunm 2) is 
meant to be a qualitative measure, scaled from I to 5, and represents a 
judgment on the part of the evaluatoron how we// the criterion is met, with 
I meaning "poorly" and 5, "excellently " Under both, the abbreviation CJ 
stands for "cannot judge " The expanded response scale offers these 
advantages: First, the scales provide more information than a binary for^ 
mat. Second, the "how well done?" column provides a way for judging the 
effectiveness rather than the mere presence of an option. For example, it is 
quite possible to have hints and explanations in explicit learning software 
that are copious but not particularly helpful, and this form is designed to 
indicate that information explicitly. 

Going through the evaluation form will reveal that in come cases the 
judgments in the second part are unnecessary. For example, if the reviewer 
circles "0" for an item under the "Degree- column, then there is no sense 
in rating how well it was done. Further, for certain items, e.g., whether the 
material "motivates the learner to use it" (Acquisition Approaches, item 
4), the rating in the "Degree" column should correspond closely to the rat- 
ing under '*How Well Done" simply because of the type of information 
requested. Note also that because scales alone may be misleading, there is 
a short sphcz for comments after each item. 

The third section of the form. Other Pedagogical Considerations, pro- 
vides for r more open-ended response in three areas. The first question 
simply asks the evaluator to make a judgment about what approach the 
courseware under review represents most dearly. Note that because the 
principles may differ in importance to a given approach or to a P^^" 
tioner of that approach, simply averaging the numbers in sections I and li 
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IS unhkely to offer a reliable numerical index that defines approach 
Rather the rater will have to determine the answer to this question 
through a careful inspection of the categories themselves and an assess- 
ment of the relative weight of each item. Question 2 in section III covers 
the relationship between the software and the syllabus as outlined in the 
previous seaion. Question 3 asks for a brief description of the methods 
and techniques the software uses and an evaluation of how weU they have 
been adapted to the computer. 

It should be noted that this supplementary form is designed for CALL 
software that includes language content and is inappropriate without sig- 
nificant changes for evaluating word-processing programs and the like 
Unauy there are judgments requested that may be duplicated in other 
parts of a more general checklist. This is a minor annoyance, however 
and perhaps not much of a price to pay for seeing the information spe- 
cific to a language-teaching approach represented in a more readily 
interpretable fashion. ' 



A Procedure for Using the Evaluation Form 

As mentioned in the opening section of this chapter, evaluating software 
for second-language instruction is a chaUenging task. The paragraphs that 
foUow offer a procedure lo aid an evaluator in completing that task sue- 
cessfuUy; its goal is to lead the evaluator to an informed decision about 
adopting a given piece of software. 

Before beginningthe evaluation process, it is necessary to acquire agen- 
eraJ evaluauon form like that in Figure 1 , to which some version of Figure 
2 above should be appended. The base form should include at least ques- 
tions about the skiU area, intended level, and hardware requirements as 
well as whether the software requires or allows for student collaboration 
(see Wyatt, Chapter 4 in this volume), etc. As suggested earlier, the 
evaluator may also wish to adapt the Supplementary Evaluation Form in 
Mgure 2 so tiiat it includes only those areas deemed relevant for the class 
or program for which the software is being considered. For instance if the 
sollware is to be used in a class taught strictly within a given approach 
then the evaluator may not feel it is necessary to rate the fit of Uie software 
with other approaches and simply eliminate those categories. When the 
iv?i™« M. encompasses a wider variety of approaches, the 
evaluator wiU need to use the entire checklist. 

The actual evaluation process involves five steps, moving from a cur- 
sory level to a very detailed and critical review. As the goal is not 
necessarily to complete the form but to decide on whether to adopt a given 
piece of software, the evaluation procedure should normally stop at any 
point where tiie evaluator becomes convinced tiiat the software is not 
appropnate for his or her class or program. q m 
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The first step is to examine the available documentation. The evaluator 
looks for general information such as the following: 

1 . language skill area and approximate level 

2. form and content of the language material 

3. t>pes of student response required (multiple choice, completion, etc.) 

4. types of feedback to the student available from the program 

5. procedures for monitoring student progress and consequent adapta- 
tion of the lesson sequencing 

6. record-keeping options available to the teacher 

The next step is to skim through selected parts of the disk for additional 
information in the areas just mentioned and to get a sense of screen layout, 
clarity of instructions, and ease of use. A good strategy is to look at parts of 
at least three lessons taken from the beginning, middle, and end of the les- 
son sequence (if one exists). In multiple-disk packages this may mean 
looking at one lesson on each of three disks. 

The third step is to take an in-depth look at one or more lessons (or parts 
of lessons if they are long). If more than one exercise type is available, then 
a sampling should be taken of each. At this stage, aU the available user 
options should be explored. Hints and help options should be tried, and 
both correct and incorrect answers should be attempted with the 
evaluator noting the computer's responses and judging their appropriate- 
ness for the presumed users. It is at this time that many of the questions on 
the Supplementary Evaluation Form can be answered with a fair degree of 
reliability, and if the evaluator*s time is an important consideration, a 
decision to adopt or reject the software can often be made. The informa- 
tion gathered from this and the previous step is also useful for 
determining whether the students will be able to use the software in its 
present form or will need additional documentation or instruction. 

The final two steps are taken when additional information is either 
needed or desired: The fourth step is to move through the entire program 
from start to finish, essentially following the same procedure as in step 
three. The fifth step, which may be accomplished without going through 
the fourth, is to field-test the software on one or more students in the tar- 
get audience. Any software that rates highly after these final two steps can 
be given a sound recommendation. 



Conclusion ^ — . 

This chapter has stressed the importance of considering teaching 
approach in the evaluation of CALL software and has presented a Supple- 
mentary Hvaluation Form to that end as well' as a procedure for 
implementing it. The supplementary form provides teachers with access 
to the kind of information that allows for a more confident decision 
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whether a given piece of software is consonant with their assumptions 
about language learning, with the needs of students in their foreign lan- 
guage classes, and with the scope and sequence of their syllabus. WhUe the 
focus of this chapter has been on the individual teacher's evaluation of 
CALL materials, there are two additional areas in which the concepts dis- 
cussed above also find relevance: software reviews and software design 

If the evaluation parameters suggested in Figure 2 are valuable to indi- 
yiduab evaluating software for their own use, they are unquestionably 
important to reviewers. Due to time factors, the complexity of the evalua- 
tion process, and the difficulty in acquiring software for that process 
pubhshed reviews play a vital role in the initial stage of deciding whether 
to consider given software for adoption or purchase. In some cases for 
expedience, published reviews may even comprise the only information 
outside of the pubhsher's description that a buyer has access to prior to 
purchasing a software package. Thus, evaluators who publish software 

reviews needtoaddressthesortsofquestions found in theSupplementary 
Evaluauon Form whether they use the actual form or not. In addition 
journals and newsletters tiiat publish reviews of CALL software could 
encourage the use of a checklist similar to tiiat in Figure 2, or at base 
develop some guidelines for reviewers as a part of tiieir editorial policy to 
ensure tiiat ^ome information concerning language-teaching approach is 
made fcxphm in the review proper. 

The implications of focusing on language-teaching approach as tiie 
organizing pnnciple in software design are also potentially quite signifi- 
cant. There are a number of specific recommendations tiiat can be made 
tosoftwareautiiorsandpublishers, aUofwhichstem from taking tiieprin- 
ciples hsted in tiie Supplementary Evaluation Form as design criteria 

First, in tiie earliest stages of tiie design process, individual software 
developers and development teams should describe tiie language-teaching 
approach to be targeted; tiiat is, determine tiie overaU lesson structure and 
the role tiiat graphics, sound, screen layout, etc., wiU play. By definition, 

SlfnTiT' V?! t"*^°' °' °' members of tiie designing team 
should have a solid background in contemporary second-language acqui- 
a^ion tiieory and research as weU as a fair amount of teaching experience 
with tiie target audience. Too often, it seems, software has been authored 
on tiie basis of general CAI design principles alone or on insufficient 
langiwge-teaching experience, with tiie result that a very limited and hap- 
nazard set of language-teaching principles is applied. 

A second implication of widespread adoption of evaluation forms that 
include tiie type of information presented in Figure 2, especiaUy in pub- 
Lft!^ir^5' " to have a significant impact on commercial 

soft^re packages. As was noted at the outset, current checklists and 
guidelines do not address specifically tiie kinds of evaluation parametere 
tound in tiie Supplementary Evaluation Form. This fact suggests that 
potenual buyers do not consider tiie areas described tiierein to be of much 
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importance, whereas checklists that include approach and strategy- 
speciflc parameters contain the message that these parameters are 
important. This could have two possible results. First, the publishers 
themselves may develop design criteria that include approach considera- 
tions, both for language software that they create directly and CALL 
software solicited from outside professionals. Second, once it is clear that 
evaluators and users consider this type of information important, pub- 
lishers may become more consistent about including it in promotional 
brochures and in the documentation accompanying the software. 

This chapter has described a rationale and a procedure to distinguish 
CALL software from other forms of CAI by something besides its 
subject-matter content. It is clear that many CALL practitioners, review- 
ers, and authors need to pay more attention to second-language 
acquisition and to teaching methodology in order to apply the insights 
from those flelds appropriately in CALL software design, evaluation, 
and use. CALL is already on the cutting edge of technology: it can also be 
on the cutting edge of methodology. In order to achieve this, it is neces- 
sary to start with the concept that second-language learning is essentially 
different from other types of learning and that, consequently, the criteria 
for judging the pedagogical soundness of instructional courseware for 
mathematics, chemistry, or history are not sufficient for judging soft- 
ware for second-language instruction. 
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